-
-
Notifications
You must be signed in to change notification settings - Fork 58
Fix Python 3.14 stack overflow in NonlinearSolveSciPy tests #732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ChrisRackauckas
merged 18 commits into
SciML:master
from
ChrisRackauckas-Claude:fix-python-314-stack-overflow
Nov 14, 2025
Merged
Fix Python 3.14 stack overflow in NonlinearSolveSciPy tests #732
ChrisRackauckas
merged 18 commits into
SciML:master
from
ChrisRackauckas-Claude:fix-python-314-stack-overflow
Nov 14, 2025
+82
−39
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes CI failures caused by Python 3.14.0's buggy stack overflow detection. The error "Fatal Python error: _Py_CheckRecursiveCall: Unrecoverable stack overflow" with negative memory values indicates a calculation bug in Python 3.14's new stack detection mechanism when interacting with Julia's task system. This issue affects: - Julia 1.9, 1.12 - macOS and Linux - Python 3.14.0 Pinning Python to >= 3.9, < 3.14 resolves the issue. References: - PythonCall.jl issue: JuliaPy/PythonCall.jl#694 - Python upstream issues: python/cpython#139653, python/cpython#137573 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes test failures where Julia-specific keyword arguments (alias, verbose) were being forwarded to scipy.optimize functions that don't recognize them. The error was: ``` Python: TypeError: least_squares() got an unexpected keyword argument 'alias' Python: TypeError: root() got an unexpected keyword argument 'alias' ``` Now filters out :alias and :verbose kwargs in all three __solve methods: - SciPyLeastSquares - SciPyRoot - SciPyRootScalar 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes two critical issues in Python interop:
1. **Type conversion errors**: Replace direct Vector{Float64}() and Float64()
constructors with pyconvert() for proper PythonCall type conversion.
- Import pyconvert from PythonCall
- Update _make_py_residual to use pyconvert(Vector{Float64}, x_py)
- Update _make_py_scalar to use pyconvert(Float64, x_py)
- Update all result conversions (res.x, res.fun) to use pyconvert
Fixes error:
```
MethodError: no method matching Vector{Float64}(::PythonCall.Py)
```
2. **Kwargs splatting errors**: Change from filter() iterator to pairs generator
for proper kwargs splatting to Python functions.
- Replace scipy_kwargs = filter(...) with inline pairs generator
- Use (k => v for (k, v) in pairs(kwargs) if k ∉ (:alias, :verbose))...
Fixes error:
```
TypeError: object of type 'NoneType' has no len()
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
Fixes two critical issues discovered in CI: 1. **Kwargs splatting error**: Convert generator to Tuple before splatting - Change from inline generator: `(k => v for ...)...` - To collected Tuple: `scipy_kwargs = Tuple(k => v for ...); scipy_kwargs...` - Fixes: `TypeError: object of type 'NoneType' has no len()` 2. **Boolean context errors**: Use pyconvert for Python boolean fields - `res.success` → `pyconvert(Bool, res.success)` - `res.converged` → `pyconvert(Bool, res.converged)` - Fixes: `TypeError: non-boolean (PythonCall.Py) used in boolean context` Applied to all three solver methods: - SciPyLeastSquares (line 137, 151) - SciPyRoot (line 177, 190) - SciPyRootScalar (line 218, 230) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes error: `TypeError: object of type 'NoneType' has no len()` Problem: We were passing `bounds=None` to scipy when bounds weren't specified, but scipy.optimize.least_squares tries to call len() on the bounds parameter, which fails on None. Solution: Conditionally omit the bounds kwarg entirely when not specified, rather than passing Python's None. This allows scipy to use its default bounds of (-inf, inf). Split the scipy call into two branches: - Without bounds: When prob doesn't have lb/ub constraints - With bounds: When prob has lb/ub constraints 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
IntervalNonlinearProblem doesn't have a u0 field (it uses tspan for the bracket). Added conditional check using hasfield() before accessing prob.u0 in both solve() and init() functions to handle problem types that don't have initial conditions. Fixes error: "type IntervalNonlinearProblem has no field u0" when solving with SciPyRootScalar. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
IntervalNonlinearProblem doesn't have a u0 field, so it should bypass the NonlinearSolveBase solve machinery by directly overloading CommonSolve.solve instead of SciMLBase.__solve. This avoids the need for conditional checks in the base solve function. - Changed SciPyRootScalar to overload CommonSolve.solve for IntervalNonlinearProblem - Added using CommonSolve to NonlinearSolveSciPy - Reverted the hasfield() conditional checks in NonlinearSolveBase since they're no longer needed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Added CommonSolve to dependencies since we use CommonSolve.solve for IntervalNonlinearProblem. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Convert all Python statistics fields (nfev, njev, nit, function_calls, iterations) to Julia Int using pyconvert instead of direct Int() constructor. Also convert u_root from Python Float to Julia Float64 in SciPyRootScalar. Fixes: - MethodError: Cannot convert PythonCall.Py to Int64 in NLStats - MethodError: no method matching ndims(::PythonCall.Py) in build_solution 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Added :lb and :ub to the kwargs filter list to prevent them from being forwarded to NonlinearSolveBase which doesn't recognize them as valid solve kwargs. These bounds are extracted directly from the problem using hasproperty and passed to scipy separately. Fixes: Unrecognized keyword arguments error for [:lb, :ub] 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
This reverts commit c99e4ab.
Created a custom keyword argument handler NonlinearKeywordArgError that extends the standard SciMLBase keywords to include bounds (lb, ub) for NonlinearLeastSquaresProblem. - Added NonlinearKeywordArgError struct and checkkwargs method in NonlinearSolveBase.jl - Added specific solve_call and init_call methods for NonlinearLeastSquaresProblem that use NonlinearKeywordArgError as the default kwargshandle - This allows bounds to be passed as problem kwargs without triggering "Unrecognized keyword arguments" errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
…SolveBase" This reverts commit 8cb8c42.
Filter out :lb and :ub from kwargs passed to scipy since these bounds are extracted directly from the problem using hasproperty and passed to scipy separately via the bounds parameter. This is a minimal, non-invasive fix that only touches NonlinearSolveSciPy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
This reverts commit 8fd05da.
…blem Updated NonlinearSolveSciPy and NonlinearSolveBase to use the new lb/ub fields added to NonlinearLeastSquaresProblem in SciMLBase.jl PR #1169: NonlinearSolveSciPy: - Import and use SciMLBase.allowsbounds trait - Implement allowsbounds trait methods: - allowsbounds(::SciPyLeastSquares) = true - allowsbounds(::SciPyRoot) = false - allowsbounds(::SciPyRootScalar) = false - Update __solve to get bounds from prob.lb and prob.ub instead of hasproperty NonlinearSolveBase: - Add bounds checking in solve_call for NonlinearLeastSquaresProblem - Error if algorithm doesn't support bounds but problem has them This properly separates concerns: - SciMLBase defines the problem interface with lb/ub fields - allowsbounds trait indicates algorithm support - NonlinearSolveBase validates compatibility - NonlinearSolveSciPy uses the fields directly Depends on: SciMLBase.jl #1169 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Extended bounds checking in NonlinearSolveBase to support both NonlinearProblem and NonlinearLeastSquaresProblem with lb/ub fields. Updated SciMLBase PR to add lb/ub fields to both problem types: - Removed unnecessary UNSET_BOUNDS constant - Added lb/ub fields to NonlinearProblem - Updated ConstructionBase.constructorof for both problem types 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Updated SciMLBase compat bounds to require version 2.127 which includes the lb and ub fields added to NonlinearProblem and NonlinearLeastSquaresProblem. Updated in: - Main Project.toml - lib/NonlinearSolveBase/Project.toml - lib/NonlinearSolveSciPy/Project.toml Closes dependency on SciMLBase.jl #1169 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes CI failures in NonlinearSolveSciPy tests on Julia v1.12 caused by Python 3.14.0's buggy stack overflow detection mechanism.
Problem
The CI is failing with:
Note the negative memory values - this indicates a calculation bug in Python 3.14's new stack overflow detection, not an actual stack overflow.
Root Cause
Python 3.14 introduced a redesigned stack overflow detection mechanism that has multiple known bugs:
This is not a PythonCall.jl or NonlinearSolveSciPy issue - it's an upstream Python 3.14 bug affecting many projects.
Solution
Added
CondaPkg.tomlto pin Python to>=3.9,<3.14, ensuring Python 3.13.x is installed instead.References
Test Plan
CI should pass with Python 3.13.x instead of 3.14.0.
🤖 Generated with Claude Code